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The  Challenge 

Interoperability  is  the  number  one  problem  in  joint  force  &  combined 

operations.  It  is  the  CINC’s  top  issue*. 

The  problem  may  be  getting  worse 

•  Real-world  operations,  evaluations  and  exercises  continue  to 
highlight  joint/combined  warfighting  capability  shortfalls 

•  As  new  coalition  partners  develop,  complex  systems  are  acquired, 
and  “fixes”  to  past  problems  are  applied  in  stove-piped  fashion 

•  Joint  Vision  2010  and  2020  call  for  increasingly  network-centric 
warfare,  dependent  upon  fully  interoperable  systems 


*  As  stated  by  Ms.  Robin  Quinlan,  Deputy  Director,  Systems  Interoperability,  Office  of  the  Secretary 
of  Defense  [Quinlan,  2000]. 
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Common  Measurement  Questions 

Are  we  able  to  identify  the  root  causes  of  interoperability 
problems? 

Are  new  system  acquisitions  becoming  more  effective  at 
avoiding  the  same  types  of  interoperability  problems  that 
occurred  yesterday? 

How  does  one  quantify  interoperability  in  an  actionable  way? 

How  do  we  measure  the  tradeoffs  between  systems 
interoperability  and  other  fundamental  attributes  of  C4I  systems 
including 

•  Security  •  Survivability  •  Performance 

•  Availability  •  Flexibility 
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SEI  Research  Goals  In  This  Area 

Understand  the  state  of  the  practice  for  measuring  systems 
interoperability 

•  Typical  approaches  for  assessing  and  measuring 
interoperability 

•  Innovations  that  are  currently  being  explored  or  piloted 
for  improving  the  state  of  the  practice 

Identify  potential  measures  and  validate  their  usefulness 
through  collaborative  field-based  investigations 

Share  the  research  results  with  the  community 
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State  of  the  Practice  Report 


•  Based  on  previously  published  reports 

•  Defines  interoperability  and  surveys  the  issues  involved  with 
achieving  interoperability 

•  Reviews  current  approaches  to  the  interoperability  problem 

•  Highlights  a  promising  new  approach  to  assessing  and 
measuring  interoperability  -  the  Levels  of  Systems 
Interoperability  (LISI)  Model 

•  Reviews  other  potential  measures  for  measuring  various 
dimensions  of  interoperability 

•  Recommends  an  initial  set  of  measures  for  improving 
interoperability 

White  paper  is  available  at  http://www.psmsc.com/ 
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Interoperability  Defined 


di\ IU 


tM  i  w - 

Operational  interoperability  Joint  Publication  1-02 
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Technical  Vs.  Operational  Interop 
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Interoperability  -  Three  Views 


Operational 

Technical 

View 

memm^ 

View 

Prescribes  Standards  and  Conventions 


Systems 

View 

Relates  Capabilities  and  Characteristics 
to  Operational  Requirements 

Adapted  from  [Chatfield  98] 
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Presentation  Outline 


Approaches  to  Measuring  Interoperability 

•  Scorecard  Approach 

•  Levels  of  Information  Systems  Interoperability  (LISI) 

•  Management  Performance  Measures 
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Technical  Interoperability  Scorecard 


System 

Compliance* 

Si 

■ 

S2 

■ 

S3 

G 

s4 

■ 

S5 

■ 

S6 

■ 

• 

* 

S„ 

■ 

Adapted  from  [Committee  99] 


+  The  entries  rate  as  pass/marginal/fail  (green,  yellow,  or  red)  the  compliance  of  systems 
S15  S2 ...  Sn  with  the  relevant  standards  and  guidance. 

©  2003  by  Carnegie  Mellon  University  1 1 


Software  Engineering  Institute 


Systems  Interoperability  Scorecard 


Systems 

View 


The  entries  rate  as  pass/marginal/fail  (green,  yellow,  or  red)  the  pairwise  interoperability  of 
the  systems  indicated  in  the  row  and  column  headings. 
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Operational  Interoperability  Scorecard 


Operational 

View 


The  diamonds  rate  as  pass/marginal/fail  (green,  yellow,  or  red)  the  ability  of  the  systems 
(indicated  as  circles)  to  provide  the  required  information  flows  (indicated  by  arrows)  for  a 
particular  mission  slice. 
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Presentation  Outline 


Approaches  to  Measuring  Interoperability 
•  Levels  of  Information  Systems  Interoperability  (LISI) 
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Software  Engineering  Institute 

Levels  of  Information  Systems 
Interoperability  [LISI] 


•  Project  initiated  by  MITRE,  The  C4ISR 
Integration  Task  Force,  and  the  C4ISR  Architecture 
Working  Group 

•  LISI  is  a  reference  model  and  process  for  assessing  information 
systems’  interoperability. 


It  provides  a  discipline  for  defining, 
measuring,  assessing,  and  certifying  the 
degree  of  interoperability  required  or 
achieved  between  systems. 


©  2003  by  Carnegie  Mellon  University 
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Level  0:  Isolated 

Manual  gateway 
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Level  1 :  Functional 

Email,  FM  voice,  tactical  data  links,  text  files 


Software  Engineering  Institute 

Level  2:  Functional 

Annotated  imagery,  maps  w/  overlays 


Software  Engineering  Institute 

Level  3:  Domain 

Common  Operational  Picture 

Warfighter  #3 


^  f^Al 


Battle  Manager 
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Level  4:  Enterprise 

Event-triggered  global  database  update 
Pentagon 


Homeland  Security 
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LISI  Maturity  Levels  -  Summary 

□ 

Enterprise 

•  Cross-domain  information  &  advanced  collaboration 

•  Interactive  manipulation  of  shared  data  &  applications 

0 

Domain 

•  Shared  data  but  separate  applications 

•  Sophisticated  collaboration 

a 

Functional 

•  Minimal  common  functions;  separate  data  &  applications 

•  Heterogeneous  product  exchange 

•  Basic  collaboration 

□ 

Connected 

•  Electronic  connected;  separate  data  &  applications 

•  Homogeneous  product  exchange 

□ 

Isolated 

•  Non-connected 

•  Homogeneous  product  exchange 

Software  Engineering  Institute 


LISI  Capabilities  Model 


Description 

Computing 

environment 

Level 

Enterprise 

Universal 

* 

Domain 

Integrated 

3 

Functional 

Distributed 

2 

Connected 

Peer-to- 

Peer 

1 

Isolated 

Manual 

0 
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The  PAID  Attributes 

P  Policies  and  procedures  that  enable  systems  to  exchange 
*  information  capabilities  and  services 

•  Standards  •  Management 

•  Security  Policy  •  Operations 


The  set  of  applications  that  enable  information  exchange, 
processing,  or  manipulation  (based  on  user  requirements). 


The  infrastructure  required  to  support  the  systems  operations 

•  Communications  and  Networks  •  Hardware 

•  System  Services  •  Security  Equipment 


D 


The  data  and  information  structures  used  to  support  both  the 
functional  applications  and  system  infrastructure 


©  2003  by  Carnegie  Mellon  University 
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LISI  Capabilities  Model 


Description 

Computing 

environment 

Level 

Enterprise 

Universal 

* 

Domain 

Integrated 

3 

Functional 

Distributed 

2 

Connected 

Peer-to- 

Peer 

1 

Isolated 

Manual 

0 
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LISI  Capabilities  Model 


Level 

(Environment) 

Interoperability  Attributes 

P 

A 

1 

D 

Enterprise 

Level 

(Universal) 

4 

c 

Multi-National 

Enterprises 

Interactive 
(Cross  applications) 

Multi- Dimensional 
Topologies 

Cross -Enterprise 
Models 

b 

Cross  Government 
Enterprises 

a 

DoD  Enterprises 

Full  Object 

Cut  and  Paste 

Enterprise 

Models  ; 

Domain 

Level 

(Integrated) 

3 

c 

b 

a 

Domain 

(Service/  Agency 
Doctrine,  Procedures, 
Training) 

Shared  Data 

(e.g.,  Simulation 
Displays,  Direct  DB 
exchange) 

DBMS 

Group  Collaboration 

Full  Text  Cut  &  Paste 

1 

Domain  Models 

1 

Functional 

Level 

(Distributed) 

2 

c 

b 

Common  Operating 
Environment 

(e.g.,  DliCOE  Level  5 
Compliance) 

Web  Browser 

Basic  Operations 

(Documents, 
Spreadsheets, 
Pictures,  ect.) 

LAN 

Program  Models  and 
Advanced  Data 
Formats 

a 

Program 

(Standard 

Procedures,  Training, 
etc.) 

Advanced 

Messaging 

(Message  Parsers, 
Email  w/  attachments) 

Connected 

Level 

(Peer-to-Peer) 

1 

d 

c 

Standards 

Compliant 

(e.g.,  JTA) 

Basic  Messaging 

(e.g.,  Simple  Text)) 

Data  File  Transfer 

Two-Way  Basic  Data  Formats 

b 

a 

Security  Profile 

Simple  Interaction 

(e.g.,  Telemetry, 
Remote  Access, 
Voice,  Fax) 

OneWay 

Isolated 

Level 

(Manual) 

0 

d 

Media  Exchange 
Procedures 

N/A 

Removable  Media 

Media  Formats 

c 

b 

a 

Manual  Access 
Controls 

Manual  Re  Entry  Private  Data 

1 

0 

No  known  interoperability 

Example 
Implementation 
Options  Table 


WAN 

•  SIPRNET 

•  JWICS 

•  NIPRNET 

•  (Internet) 

•  DISNLES 

•  VSAT 

•  DISN 


NET 

•  Link16 

•  Link  22 

•  UHF  Radio 

•  VHF  Nets 

•  Ethernet 

•  Token  Ring 

•  Other  Nets 
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The  LISI  Metric 


Operational 

Technical 

View 

.e 

View 

M 

Generic  '  ^  j 

Expec 

Systems  View 

Soecific  o 

~ — “ntional  View 

L 

vorise  4 

\ 


C> 

) 

L 

•late 

Sublevel  es  fc 

defined 
through 

Systems 

View 
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Example  LISI  Profile  &  Resulting 
Metric  for  Single  System 


Technical 

View 


Level 

(Environment) 

Interoperability  Attributes 

P 

A 

1 

D 

Enterprise 

Level 

(Universal) 

4 

c 

b 

a 

Domain 

Level 

(Integrated) 

3 

c 

Service-approved 
MNS  &  ORD, 

WAN  addressing 

TCP/IP  WAN, 

NFS, 

SNMP, 

ISDN  card 

MIDB,  SQL 

b 

a 

scheme 

Functional 

Level 

(Distributed) 

2 

c 

b 

Dll  COE  Compliant. 
Windows-std  file 
name  extensions 

*  *l?4.0  *  ' 

MS  Office,  Access 
CMTK,  5D, 

MPEG  Viewer 

IPLAN 

NES 

NTP.X.500 

'  *  NIFT,*  * 

USMTF, 

x.400, 

.wks,  .xls, 

DTED,  DBDB, 

.ppt,  .doc, 

RPF,  CGM,  JBIG, 
JPEG,  HTML,  VPF 

a 

On  line 

Documentation 

Eudora 

TBS, 

LINK  16  &  22 

Connected 

Level 

(Peer-to-Peer) 

1 

d 

Windows  Interface 
Design  Guide 
(JTA) 

HF  Data  Modem, 
Kermit,  STU  III, 

GSM  Cellular 

MPEG  1.2 

GKS,  wmf 

c 

FTP 

b 

ITU-T  Rec  X.509.  Chat  2.0 

a 

Mil  Std  2045-28500 
Security  Labels 

Win32  API.PPS 

GBS 

Isolated 

Level 

(Manual) 

0 

d 

Login  procedures 

c 

1 

b 

a 

0 

No  known  interoperability 

System 
generic 
metric  is 
“G2c” 
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When  Comparing  Two  Systems 

The  LISI  metric  is  simply  the  lesser  of  the  two  systems’ 
generic  levels.  This  is  the  expected  LISI  metric. 


System  A 


System  B 


Expected  (A&B)  E3a 
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Two  Systems  in  Operational  Use 

This  LISI  metric  takes  into  account  the  environmental 
factors  and  specific  mission  requirements.  This  is  the 
specific  LISI  metric. 


©  2003  by  Carnegie  Mellon  University 
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Operational  Detailed  Measures  of 
Interest  -  Examples 


0*  Connectivity 


Qeff  ~  (2max  2o/j)X(^/  * p) 


Where: 


Capacity 


Qeff  =  Effective  system  capacity  (data  rate) 

Qmax  =  Maximum  data  rate 

Qc*  =  System  overhead  data  rate 


0*  System  Overload 
0^  Underutilization 
0^  Undercapacity 


•  Percentage  of  initial  transmission  messages 
received  correctly  by  shooters 

•  Percentage  of  consistency/disparity  of  redundant 
data  sources 

•  Number  of  tries  needed  to  establish  connections 

•  Delay  in  sending  critical  command  messages 
and  time  to  receive  acknowledge  messages 


01  Data  latency 


Information  interpretation  &  utilization 
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Capabilities  Model 


* 


Generate  system 
profile  using  LISI 
questionnaire 


System  profile  and 


Technology  Insertion 
Forums 
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Other  system  profiles 
with  corresponding 
LISI  metric 


i 


LISI  Data 
Repository 
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Presentation  Outline 


Approaches  to  Measuring  Interoperability 


Management  Performance  Measures 
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Measuring  Management  Commitment 
to  Interoperability 

The  Committee  to  Review  DoD  C4I  Plans  and  Programs  found 
that: 

“achieving  C4I  interoperability  is  more  a  matter  of 
organizational  commitment  and  management  than  one  of 
technology” 


Potential  management  measures 

•  Number  of  systems  certified  to  be  interoperable 

•  Time  (or  personnel  required  to  develop  time-phased  force  and 
deployment  data 

•  Time  need  to  stand  up  a  tactical  network  for  a  joint  task  force 

•  Number  of  individuals  trained  in  the  use  of  specific  C4I  systems 


Software  Engineering  Institute 


Next  Steps 


Q  Establish  collaborative  relationships  with  stakeholders 
who  are  conducting  interoperability  assessments. 

Q  Provide  guidance  for  measurement  aspects  of  the 
assessment  process(es). 

O  Pilot  the  process  using  measures  developed  in  stage  2. 

(||  Conduct  a  lessons  learned  to  evaluate  the  utility  of  the 
measures  that  were  piloted. 

Q  Assess  the  results  of  the  pilot  study,  develop 
recommendations  and  publish  the  results  for  the 
community. 
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